Agreement between pulse oximetry and arterial oxygen saturation measurement in critical care patients during COVID-19: a cross-sectional study

Some publications suggest that pulse oximetry measurement (SpO2) might overestimate arterial oxygen saturation (SaO2) measurement in COVID-19 patients. This study aims to evaluate the agreement between SpO2 and SaO2 among COVID-19 and non-COVID-19 patients. We conducted a multicenter, prospective study including consecutive intensive care patients from October 15, 2020, to March 4, 2021, and compared for each measurement the difference between SpO2 and SaO2, also called the systematic bias. The primary endpoint was the agreement between SpO2 and SaO2 measured with the Lin concordance coefficient and illustrated using the Bland and Altman method. Factors associated with systematic bias were then identified using a generalised estimating equation. The study included 105 patients, 66 COVID-19 positive and 39 COVID-19 negative, allowing for 1539 measurements. The median age was 66 [57; 72] years with median SOFA and SAPSII scores of, respectively, 4 [3; 6] and 37 [31; 47]. The median SpO2 and SaO2 among all measurements was respectively 97 [96–99] and 94 [92–96] with a systematic bias of 0.80 [− 0.6; 2.4]. This difference was, respectively, 0.80 [− 0.7; 2.5] and 0.90 [− 0.3; 2.0] among COVID-19 positive and negative patients. Overall agreement measured with the Lin correlation coefficient was 0.65 [0.63; 0.68] with 0.61 [0.57; 0.64] and 0.53 [0.45; 0.60] among the COVID-19 positive and negative groups, respectively. Factors independently associated with the variation of the SpO2-SaO2 difference were the PaO2/FiO2 ratio and need for mechanical ventilation. In our population, agreement between SpO2 and SaO2 is acceptable. During the COVID-19 pandemic, SaO2 remains an efficient monitoring tool to characterise the level of hypoxemia and follow therapeutic interventions. As is already known about general intensive care unit patients, the greater hypoxemia, the weaker the correlation between SpO2 and SaO2. Supplementary Information The online version contains supplementary material available at 10.1007/s10877-022-00959-2.

Oxygenation during ARDS is one of the most important parameters to monitor. It can be done non-invasively by pulse oximetry to measure peripheral oxygen saturation (SpO 2 ), which reflects the arterial oxygen saturation (SaO 2 ), or invasively by arterial blood sampling to measure arterial oxygen pressure (PaO 2 ). PaO 2 is related to SaO 2 by the haemoglobin dissociation curve with a sigmoid relation [5], and the ratio of PaO 2 to the inspired fraction of oxygen (FiO 2 ), also named PaO 2 /FiO 2 , is widely used to report the pulmonary capacity to oxygenate blood and therefore tissues [6]. A less invasive and simpler way of monitoring oxygenation capacity is to monitor SpO 2 [7][8][9], which is particularly useful in guiding daily care as well as in judging the need for therapeutic interventions such as oxygen titration or monitoring in critical and acute situations. SpO 2 /FiO 2 is, similarly to PaO 2 /FiO 2 , used to classify patients in studies [10][11][12][13] when PaO 2 is not available.
Numerous studies have evaluated the agreement between SpO 2 and SaO 2 [14][15][16]. A strong correlation is reported with a bias between − 0.70 to + 1.86% [16]. Many factors have been reported to decrease the reliability of this measurement, such as hypoxemia, anaemia, hyperbilirubinaemia, skin pigmentation, low flow states or hypothermia [17][18][19][20]. Since the onset of the COVID-19 pandemic, some doubts have been raised about the quality of the agreement between SpO 2 and SaO 2 among COVID-19 patients. Wilson-Baig et al. published a retrospective, monocentric study of 17 intensive care unit patients and documented an average underestimation of the SaO 2 by the SpO 2 of 5.3% [21]. These findings triggered the hypothesis that a COVID-19 infection could influence the agreement between SaO 2 and SpO 2 through mechanisms including microvascular complications or variations in plasma protein levels, such as D-dimer, interleukin or ferritin, and that interacting with haemoglobin could change its spectral properties between 660 and 940 nm [22]. Nevertheless, the lack of a control group in the Wilson-Baig study limited the significance of its findings. Because SpO2 is a key component in the management of acute respiratory failure worldwide, especially in times of crisis and in emerging countries, we therefore conducted a study to investigate SpO2-SaO2 concordance in consecutive ICU patients, including COVID-19 positive and negative patients.
We thus conducted a study that aimed to investigate SpO 2 -SaO 2 agreement among consecutive intensive care unit patients, including COVID-19 positive and negative patients.

Study population
We conducted a prospective multicentric, observational study, including consecutive patients admitted in intensive care units (ICUs) with both COVID-19 and non-COVID-19 patients. Patients were admitted to five ICUs located in two different centres: four ICUs in La Pitié-Salpêtrière Hospital (Paris, France) and one ICU in Ambroise-Paré Hospital (Boulogne-Billancourt, France). All patients admitted in the five ICUs were consecutively included between October 15, 2020, and March 4, 2021, without exclusion criteria. The study was approved by the ethics committee of the French Society of Anaesthesiology and Intensive Care (IRB number 00010254-2021-103). Accordingly, with the European General Data Protection Regulation, we obtained an agreement of Assistance

Endpoints
The primary endpoint was the agreement between SpO 2 and SaO 2 , analysed according to the method of Bland and Altman [24]. We defined the systematic bias of this agreement as the difference between SpO 2 and SaO 2 .
Secondary endpoints were the concordance between SpO 2 and SaO 2 and the identification of factors independently associated with systematic bias.

Data collection
This study used only routinely collected data. All existing pairs of SpO 2 and SaO 2 measurements were collected during the ICU stay within a limit of 30 days after ICU admission. The arterial sampling used to measure SaO 2 was performed either by direct arterial puncture or by sampling from an arterial catheter line located in the radial or femoral position. The samples were analysed using a blood gas analyser (ABL90 FLEX, Radiometer, Denmark) and recorded. Just before each arterial sampling, the exact time was assessed and SpO 2 was collected from the monitor. We thus used electronic medical record that report SpO 2 min by min to confirm the measurement done at bedside. SpO 2 measurement were done with the type of sensor available at the bedside (Supplementary material 2).
Concomitantly with SaO 2 measurement, each arterial sample was used to measure PaO 2 , PaCO 2 , pH and lactate. At the time of each SaO 2 -SpO 2 pair measurement, we also collected FiO 2 , temperature, haemoglobin and haematocrit, type of ventilation/oxygenation and patient position (supine or prone). The use of vasopressor, nitric oxide or corticosteroid therapy was also reported. Some biological parameters were collected when available, such as ferritin, D-dimer, fibrinogen, bilirubin and C-reactive protein.

Statistical analysis
We considered the SaO 2 to be the reference measurement. The systematic bias (defined as the SpO 2 -SaO 2 difference) was described using the Bland and Altman method and reported with a mean measurement. Precision was reported with a 95% Bland-Altman Limit Of Agreement (LOA) [24]. The Bland-Altman method has been reported to sometimes provide biased estimates especially when the variances of the measurement errors of the two methods are different [25][26][27]. For this reason, we carried out the Taffé method using R statistical software package "MethodCompare" [28].
We described the relation between SpO 2 and SaO 2 using the Lin correlation coefficient for repeated data [29] and used a random-effect model (GEE [generalised estimating equation] type) to estimate the effect of different confounding factors on systematic bias while accounting that repeated measurements were made for several patients [30,31]. Candidates' variables included in the GEE model were selected when p-values were lower than 0.1 in univariate analysis and if considered important by the authors (AJ, JMC) according to their clinical expertise. We then conducted a stepwise backward regression and retained factors independently associated with a p-value lower than 0.05. Continuous variables were described using the median and the first and third quartiles, while categorical variables were described using relative number and percentage. We did not impute missing data except for missing haemoglobin at admission (n = 6), for which the mean between the previous and the next measurement was imputed. If those measurement was not available, if the missing data was the first of the series for example, we imputed the missing data with the closest in time. Data were analysed using R v4.0.2 (R-project.org, the R foundation for statistical Computing, Vienna, Austria).
In this study, 1239 blood gazes were matched with relevant clinical information measured at the time of the sampling. Among these samples, 301 were collected from COVID-19 negative patients while 1238 were collected from COVID-19 positive patients. From these 1539 blood samples, the mean PaO 2 /FiO 2 was 250 [188-330] and 130 [90-178], respectively, for COVID-19 negative and positive patients. Information at the blood sample level is reported in Supplementary material 3.

Primary endpoint
The mean systematic bias was of 1.0% among all included patients with a 95% LOA [− 5.0; 7.1], highlighting that SpO 2 overestimated SaO 2 by 1.0% (Fig. 1A). The subgroup analysis identified a mean systematic bias of respectively 1.0% 95% LOA [− 5.3; 7.4] (Fig. 1B) and 1.0% 95% LOA [− 3.6; 5.6] (Fig. 1C) for COVID and non-COVID patients, respectively. These findings with the median (Q1-Q3) and the mean (95% CI) are summarised in Table 2. We report in Supplementary Material 4 the Bland-Altman plot resulting from the Taffé et al. approach and conducted to address the differences in variances between the measurement errors of the SpO 2 and the SaO 2 .  (Table 3). Among COVID-19 patients, these two variables were also independently associated with an increase in systematic bias (p < 0.01, Supplementary material 7).

Secondary endpoints
The time since the first measurement had a statistically significant (p < 0.01) though clinically non relevant impact on the systematic bias with an increase of 0.05% for each supplementary day (Supplementary Material 8).  COVID-19 infection to be associated with a lower agreement between SpO 2 and SaO 2 [21,32]. These studies were both retrospective, including a limited number of patients and without control groups. They mainly relied on two hypotheses. The first was that the major inflammatory state of COVID-19 patients requiring ICU care sustained a significant increase in blood concentration of proteins such as D-dimer or ferritin. It has been suspected that such inflammatory proteins could have absorption properties affecting the pulse oximeter signal, consequently reducing measurement accuracy [22]. The other hypothesis, supported by nonpeer-reviewed publications, is that the virus would directly bind to haemoglobin and modify infrared absorption [33]. Our prospective study, with a larger cohort of patients and including a control group, did not confirm these preliminary results. We suggest that differences observed between SaO 2 and SpO 2 might be explained by the severity of hypoxemia rather than by COVID-19. On the other side, a recent study reported, among COVID-19 patients, a 0.05% bias with a limit of agreement ranging from − 2.2 to 2.3 which is consistent with the results presented in our study even if we report a wider limit of agreement [14]. The impact of hypoxemia on the measure of SpO 2 is a well-known issue [34]. It has been reported that the poor quality of the measure of hypoxemic patients could be related 1/to the arteriolar dilation caused by tissue hypoxia that induces venous pulsation, which reduces the quality of the measure [9,35], 2/with the quality of the pulse oximeter being used [35,36] or high levels of skin pigmentation [14,37,38].
Several studies have reported low agreement between SpO 2 and SaO 2 among ICU patients [15,20,35,39] with contradicting results: some report that SpO 2 overestimates SaO 2 , and others report the opposite. In our study, we observed that SpO 2 minimally overestimated SaO 2 with a mean systematic bias of 1.0% 95% LOA [− 5.0; 7.1].
This LOA spread implies that even with a good agreement and concordance with SaO 2 , SpO 2 remains a daily routine monitoring device that needs to be confirmed by a SaO 2 measurement whenever needed and especially when a precise measurement is required. This issues has been known for a long time [34] and a study, conducted in 2001, reported similarly a 2.1 standard deviation of the mean difference between SpO 2 and SaO 2 . This study also reports that the type of oximeters, the presence of hypoxemia and the requirement for vasoactive drugs might influence SpO 2 measurements [15]. To address this uncertainty, this study suggests that a SpO 2 above 94% is necessary to ensure a SaO 2 superior to 90%.
Our study has limitations. First, we conducted an observational study including COVID-19 positive and negative patients with significant differences both in baseline characteristics and outcomes, especially regarding hypoxemia. To account for these differences and allow for meaningful between-group comparisons, we proposed a multivariate analysis accounting for repeated measurements. One solution to address this issue would have been to include more hypoxemic and severely ill patients in the COVID-19 negative groups-those with ARDS, for instance.
Second, this study did not allow for an exploration of the hypothesis that inflammatory biological markers could influence systematic bias. These markers, such as D-dimer, fibrinogen, bilirubin, ferritin or C-reactive protein, were indeed inconstantly measured among COVID-19 positive patients and rarely measured among COVID-19 negative patients. Other factors, such as skin colour, methemoglobinemia, carboxyhaemoglobinia or hyperbilirubinemia, were also not considered.
Third, while invasive mechanical ventilation allows for accurate FiO2 measurement, other modes of oxygenation may not allow for such accuracy. In a modeling article, Wagstaff et al. report that HFO is the only oxygenation  [40]. Indeed, FiO2 decreases when the respiratory rate increases. Finally, we did not analyse the impact of the type of pulse oximeters used. Devices are known to cause substantial differences in bias and precision, especially at low saturation [34] and the COVID-19 surge led hundreds of manufacturers proposing new devices with sometime questionable standard quality. This reinforces the importance for caregivers to ensure a careful attention the devices selected in their wards especially when caring for hypoxemic patients.

Conclusion
This study confirms existing data about the reliability of SpO 2 measurement in ICUs and suggests that possible differences between the COVID-19 positive and negative populations might be related to the higher prevalence of severe hypoxemia among COVID-19 positive patients'. In clinical practice, it remains important to acknowledge that SpO 2 measurement reliability worsens with hypoxemia severity. Further studies are needed to explore the potential impact, if any, of inflammatory proteins on the accuracy of SpO 2 .
Author contribution AJ, MP, AVB, JMC conceived and designed the study. AJ, MP, MP extracted data. AJ performed the analyses. AJ, AJ, MP, AVB, JMC contributed to the interpretation of the results and wrote the first draft of the manuscript. All authors acknowledge full responsibility for the analyses and interpretation of the report. All authors have read and approved the final manuscript. JMC is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding Not applicable.
Data availability Relevant data can be available after reasonable request to the corresponding author and discussion with the working group.

Declarations
Conflict of interest A-J reports personal fees from LFB for an educational project outside of the submitted work. JM-C reports personal fees and non-financial support from Drager, GE Healthcare, Sedana Medical, Baxter, and AOP Health; personal fees from Fisher and Paykel Healthcare, GSK, Guilead, Orion, Philips Medical, and Fresenius Medical Care; and non-financial support from LFB and Bird Corporation, outside of the submitted work. M-P reports no competing financial or non-financial interests. AV-B is the recipient of a research grant from GSK, outside of the submitted work. AV-B is member of an advisory board for Air Liquide Healthcare.
Ethical approval and consent to participate. This study was approved by the ethics committee of the French Society of Anaesthesiology and Intensive Care (IRB number 00010254-2021-103). In accordance with French law, written consent was not required, but the non-opposition of the patient or his relatives to the use of his data was recorded before discharge from ICU.

Consent for publication Not applicable.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.